Applying Unsupervised Learning To Support Vector Space Model Based Speaking Assessment

Vector Space Models (VSM) have been widely used in the language assessment field to provide measurements of students’ vocabulary choices and content relevancy. However, training reference vectors (RV) in a VSM requires a time-consuming and costly human scoring process. To address this limitation, we applied unsupervised learning methods to reduce or even eliminate the human scoring step required for training RVs. Our experiments conducted on data from a non-native English speaking test suggest that the unsupervised topic clustering is better at selecting responses to train RVs than random selection. In addition, we conducted an experiment to totally eliminate the need of human scoring. Instead of using human rated scores to train RVs, we used used the machine-predicted scores from an automated speaking assessment system for training RVs. We obtained VSM-derived features that show promisingly high correlations to human-holistic scores, indicating that the costly human scoring process can be eliminated.